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CONSTRUCT  VALIDATION  OF  BUSINESS  ECONOMIC  PERFORMANCE  MEASURES 
A  STRUCTURAL  EQUATION  MODELINC  APPROACH 


ABSTRACT 
A  structural  equation  modeling  approach  is  employed  to  assess  the 
measurement  properties  of  the  business  economic  performance  construct, 
Data  on  three  dimensions  of  performance — sales  growth,  net  income 
growth,  and  profitability  (ROI) — were  collected  using  two  different 
methods — (i)  perceptual  assessments  of  senior  executives;  and  (ii) 
secondary  data  sources.   The  analysis  indicate  that  convergent  and 
discriminant  validity  were  achieved  only  when  systematic  sources  of 
variation  (method  factors)  were  considered.   Ihe  advantages  of  this 
approach  in  relation  to  the  commonly-used  MTMM  framework  are  high- 
lighted, and  implications  for  strategy  researchers  are  noted. 


An  earlier  version  of  this  paper  has  been  accepted  for  presentation 
at  the  45tn  Annual  meeting  of  the  Academy  of  Management,  San  Diego, 
August  1985.   Patrick  Caughan ' s  assistance  in  data  collection  is 
greatly  appreciated. 
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Organizational  economic  performance  or  a  broader  concept  of  organi- 
zational effectiveness  is  fundamental  to  both  descriptive  and  prescrip- 
tive research  in  many  management  disciplines,  including  organization 
theory  and  strategic  management.   In  addressing  this  theme,  researchers 
have  adopted  a  wide  array  of  conceptualizations  and  operationalizations 
depending  upon  their  main  research  question,  their  disciplinary  focus, 
and  data  availability.   A  review  of  the  research  literature  on  the 
complex  topic  of  organizational  performance  is  not  attempted  here  since 
good  discussions  can  be  found  in  Campbell  (1977),  Chakravarthy  (1984), 
Ford  and  Schellenberg  (1982),  Hofer  (1983),  Kanter  and  Brinkerhoff  (1981), 
Kirchoff  (1977),  Seashore  and  Yuchtman  (1967)  and  Steers  (1975;  1977). 

BUSINESS  ECONOMIC  PERFORMANCE: 

THE  ISSUE  OF  OPERATIONALIZATION  IN  STRATEGY  RESEARCH 

Researchers  in  the  emerging  discipline  of  strategic  management  are 
centrally  concerned  with  issues  of  conceptualizing  and  measuring  organi- 
zational performance  (Schendel  &  Hofer,  1979).   However,  recognizing  the 
complexity  of  organizational  performance,  they  have  largely  focused 
their  attention  on  a  narrower  concept  of  Business  Economic  Performance 
(BEP).   Typically,  BEP  has  been  conceptualized  in  terms  of  indicators 
such  as  Return  on  Investment  (ROI),  Return  on  Sales  (ROS) ,  Sales  Growth, 
and  Price-to-Earnings  (P/E)  ratio  (see  Hofer,  1983  for  a  review  of 
various  performance  indicators  used  in  strategy  research). 

In  attempting  to  operationalize  BEP,  researchers  have  adopted  one 
of  two  methods — (a)  use  of  "secondary"  data  sources  such  as  COMPUSTAT 
(e.g.,  Ramanujam,  1984;  Schendel  &  Patton,  1978)  or  (b)  use  of  "primary" 
sources  by  requesting  managers  to  provide  perceptual  assessments  of 
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their  level  of  performance  relative  to  competition  (e.g.,  the  PIMS 
program)  or  their  level  of  satisfaction  with  performance  (Bourgeois, 
1980;  Gupta  &  Govirdarajan,  1984). 

While  researchers  typically  use  one  of  the  two  methods,  an 
encouraging  exception  is  seen  in  Dess  and  Robinson's  (1984)  study  on 
the  correspondence  between  BEP  measures  from  two  different  methods. 
Using  data  from  a  sample  of  26  units,  they  reported  a  positive  and 
significant  association  between  "self-reported  objective"  and  "subjec- 
tive" data  on  two  performance  dimensions — return  on  assets,  and  sales 
growth.   Such  an  approach  is  a  welcome  point  of  departure  since  it  cer- 
tainly enhances  the  quality  of  operationalization.   However,  their 
"methods"  are  conceptually  similar  in  the  sense  of  employing  data 
collected  from  only  primary  source  and  represent  the  "within  method" 
type  of  triangulation  (Denzin,  1978)  . 

The  limitations  of  this  type  of  triangulation  are  noted  by  Denzin, 
"Observers  delude  themselves  into  believing  that ... .different  varia- 
tions of  the  same  method  generate. .. .distinct  varieties  of  triangulated 
data.   But  the  flows  that  arise  using  one  method  remain....  (1978;  pp. 
301-302).   The  weaknesses  in  the  use  of  a  single  source  of  data  can  be 
overcome  by  employing  two  different  data  sources — viz.,  primary  and 
secondary,  which  is  in  line  with  Campbell  and  Fiske's  (1959)  call  for 
using  "maximally  different"  methods  to  assess  convergent  validity  of 
operationalization. 

Such  an  extension  was  attempted  (Venkatraman  &  Ramanujam,  1985)  by 
collecting  objective  data  from  secondary  sources  (as  opposed  to  self- 
reported  objective  data  as  in  Dess  and  Robinson's  study)  and  perceptual 
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assessments  of  top-level  managers  (primary  data).   By  treating  the  two 
data  sources  as  distinct  "methods"  within  Campbell  and  Fiske's  (1959) 
MultiTrait,  MultiMethod  (MTMM)  framework,  the  four  general  criteria  for 
convergent  and  discriminant  validity  of  each  of  the  three  performance 
dimensions — net  income  growth,  sales  growth,  and  profitability  (ROI) — 
were  broadly  satisfied. 

In  strategic  management  research,  where  attempts  at  construct 
measurement  are  not  yet  systematic  (Venkatraman  &  Grant,  1986),  use  of 
MTMM  matrix  to  assess  convergent  and  discriminant  validity  is  a  welcome 
point  of  departure.   However,  since  the  criteria  generally  employed  in 
a  MTMM  framework  are  rather  broad  and  open  to  researchers'  inter- 
pretation, this  approach  is  under  attack  (see  Bagozzi,  1980;  Schmitt, 
Coyle,  &  Saari ,  1977).   Consequently,  in  this  study  we  undertake  an 
extended  examination  of  the  construct  measurement  of  the  three  perfor- 
mance dimensions.   The  rationale  for  such  an  extension  is  that  critical 
evaluations  of  measurement  properties  enhance  the  quality  of  operation- 
alization,  which  is  essential  for  rigorously  testing  theoretical  rela- 
tionships (Schwab,  1980).   This  is  especially  important  since  some 
studies  which  satisfied  the  broad  MTMM  criteria  failed  to  turn  up  simi- 
lar results  when  the  variance  in  measurement  was  partitioned  into  its 
constituent  components  (see  Bagozzi,  1980;  pp.  136-153  for  a  discussion) 

STUDY  OBJECTIVES 
Our  purpose  in  this  paper  is  to  illustrate  the  benefits  of  adopting 
a  structural  equation  modeling  approach  (Joreskog,  1969;  Joreskog  & 
Sorbum,  1978;  1979)  to  address  a  broader  set  of  measurement  questions 
which  cannot  be  directly  addressed  within  the  MTMM  framework.   We  use 
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the  construct  of  BEP  and  data  collected  from  two  different  sources  for 
this  purpose.   The  specific  research  questions  for  this  study  are: 

1.  To  what  extent  are  convergent  and  discriminant  validity  of  BEP 
achieved  when  measurement  error  is  included  for  consideration, 
and  the  variance  in  measurement  partitioned  into  trait,  method, 
and  error  components?; 

2.  Is  the  "secondary"  method  superior  than  the  "primary"  method?  or 
vice-versa;  and 

3.  Since  convergent  and  discriminant  validity  are  critically  depen- 
dent on  the  use  of  "maximally  different  methods"  (Campbell  & 
Fiske,  1959),  are  the  two  methods  dissimilar? 

The  first  question  addresses  the  construct  validity  issues  systema- 
tically by  identifying  reasons  for  the  support  (or,  lack,  of)  for  the 
various  validity  criteria,  by  decomposing  the  measurement  variance  into 
its  various  components.   The  second  question  aims  to  identify  the  rela- 
tive superiority  of  the  two  methods,  and  the  third  aims  at  an  evaluation 
of  the  robustness  of  the  measurement  properties,  which  can  be  inferred 
when  similar  results  are  obtained  from  dissimilar  methods. 

ANALYTIC  METHOD 
This  section  discusses  the  analytical  method  employed  to  address 
the  three  research  questions.   To  begin  with  a  brief  outline  of  the 
comparative  benefits  of  the  structural  equation  model  over  the  tradi- 
tional MTMM  approach  is  presented.   Subsequently,  the  proposed  analy- 
tical scheme  is  described  with  the  measurement  equations  and  the 
analytical  procedure  for  testing  a  set  of  sequential  models. 

Comparative  Benefits  of  the  Structural  Equation 
Model  Approach  over  the  MTMM  Method 

A  major  limitation  of  the  MTMM  approach  to  construct  validation  is 

its  inability  to  partition  the  amount  of  variation  in  measurement  into 
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its  components  such  as  trait,  random  error,  and  systematic  (i.e.  method) 
error.   An  alternative  approach  (confirmatory  factor  analysis),  which 
is  based  on  Che  structural  analysis  of  covariance  matrices  (Joreskog  & 
Sorbum,  1978),  has  found  acceptance  in  relatively  mature  disciplines 
such  as  psychology,  sociology,  and  marketing  (Bagozzi,  1980;  Fornell, 
1982).   In  addition,  it  has  also  been  employed  in  strategy  research  by 
Farh,  Hoffman,  and  Hegarty  (1984)  for  assessing  the  measurement  proper- 
ties of  Hambrick's  (1981a)  environmental  scanning  scale. 

This  method,  in  addition  to  examining  convergent  and  discriminant 
validity,  can  be  used  to  assess  the  reliability  of  indicators  and  com- 
posite measures  with  less  assumptions  than  those  underlying  the  calcu- 
lation of  other  reliability  indices  (see  Bagozzi,  1981;  Fornell  and 
Larcker,  1981;  and  Werts,  Linn,  and  Joreskog,  1974).   The  comparative 
benefits  of  this  approach  are  perhaps  best  summarized  by  Kenny: 

The  application  of  confirmatory  factor  analysis 
to  the  multitrait,  multimethod  matrix  has  a  number 
of  advantages  over  the  traditional  Campbell-Fiske 
criteria:   (a)... (it)  gives  estimates  of  parameters 
while  Campbell-Fiske  criteria  are  only  rules  of 
thumb.   (b)  Significance  tests  are  possible  with 
confirmatory  factor  analysis.   (c)  Given  marked 
differences  in  the  reliability  of  measures,  the 
Campbell-Fiske  criteria  are  misleading. .. .while 
confirmatory  factor  analysis  takes  into  account 
differential  reliability  (1976;  p.  248). 

Specifically,  the  alternative  scheme  provides:   (a)  a  formal  statistic 
for  judging  the  entire  validity  of  a  construct;  (b)  an  indication  of 
the  degree  to  which  operationalizations  measure  the  concepts  they 
intend  to  measure;  and  (c)  a  decomposition  of  the  variance  in  measure- 
ment into  its  components. 
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Four  Measurement  Models  for  BEP 

We  propose  tour  measurement  models  for  BEP  based  on  Joreskog's 
general  analysis  of  covariance  structures  (Joreskog,  1969;  1971; 
Joreskog  &  Sorbum,  1973;  1979).   We  have  provided  the  required  analyti- 
cal equations  for  the  proposed  models  in  the  following  paragraphs.   But 
our  discussion  is  not  highly  technical,  and  those  readers  requiring 
more  technical  details  are  directed  to  Bagozzi  (1980),  Fornell  (1982), 
Joreskog  and  Sorbum  (1973),  and  Long  (1983). 

The  first  model  is  a  test  for  convergent  validity  with  only  the 
trait  factors.   If  the  first  model  is  not  supported,  then  it  is  extended 
by  adding  method  factors  to  evaluate  whether  the  additional  incorpora- 
tion of  systematic  sources  of  variation  provides  better  support  to  the 
second  model.   The  third  model  tests  for  discriminant  validity,  while 
the  fourth  tests  for  the  degree  of  dissimilarity  of  the  two  methods 
used.   The  analytical  equations  for  these  models  and  the  criteria  to 
be  adopted  for  model-testing  are  discussed  in  the  following  paragraphs. 

Testing  for  Convergent  Validity — Model  1.   Convergent  validity 
refers  to  the  degree  to  which  two  or  more  attempts  to  measure  the  same 
trait  through  maximally  different  methods  are  in  agreement.   Following 
Joreskog's  work,  the  basic  model  for  convergent  validity  can  be  written 
as : 

X  -  AC  +  6  (1) 

where,  X  is  a  vector  of  p  measurements,  %    is  a  k  <  p  vector  of  traits, 
5  is  a  vector  of  unique  scores  (random  errors),  and  A  is  a  p  X  k  matrix 
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of  factor  loadings.   With  the  assumptions  of  E(£)  =  U,  E(£€f)  =  $,  and 
E(56')  =  V  ,  the  variance-covariance  matrix  or  X  can  be  written  as 

v     =  Al|>A»  +  w  (2) 

where,  E  is  the  variance-covariance  matrix  of  observations,  $  is  the 
intercorrelation  among  the  traits,  and  ¥  is  a  diagonal  matrix  of  error 

variances  (9 , )  for  the  measures. 

o 

2 
Maximum  likelihood  parameter  estimates  for  A,  <f> ,  H ,    and  a  x   good- 
ness of  fit  index  for  the  null  model  implied  by  equations  (1)  and  (2) 

can  be  obtained  from  the  LISREL  program  (Joreskog  &  Sorbum,  1973). 

? 
The  probability  level  associated  with  a  given  x~  statistic  indicates 

the  probability  of  attaining  a  larger  x"  value,  given  that  the  hypo- 
thesized model  holds.   The  higher  the  value  of  p,  the  better  is  the 
fit,  and  as  a  rule  of  thumb,  values  of  p  >  0.10  are  considered  as 
indications  of  satisfactory  fit  (Lawley  &  Maxwell,  1971). 

This  model  hypothesizes  that  all  the  variation  and  covariation  in 
the  measurement  of  traits  can  be  accounted  for  by  the  theoretical  con- 
cepts that  the  measurements  are  intended  to  capture  plus  random  error. 
Figure  1  is  a  diagramatic  representation  of  the  first  model  implied  by 
equations  (1)  and  (2)  in  this  study,  where  three  traits  (dimensions) 
of  performance  are  each  measured  by  two  methods. 


INSERT  FIGURE  1  ABOUT  HERE 


Testing  for  Convergent  Validity  With  Method  Factors — Model  2.   If 
the  previous  model  fails  to  achieve  a  satisfactory  fit  to  the  data, 
one  can  examine  potential  improvement  with  explicit  modeling  of  method 
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factors — "primary"  and  "secondary"  sources  of  data.   The  underlying 
rationale  for  this  model  is  that  the  observations  are  not  only  a  func- 
tion of  the  trait  and  random  error,  but  are  also  influenced  by  syste- 
matic sources  of  variation  such  as  the  source  of  data.   Two  method 
factors  are  added  to  the  first  model  as  systematic  sources  of  variation 
in  addition  to  random  variations  (i.e.,  unique  uncorrelated  errors 
represented  as  6  )•   Method  factor  1  represents  the  perceptual  source 
of  data  provided  by  the  respondents,  while  method  factor  2  represents 
the  objective  or  secondary  data  source.   A  diagramatic  representation 
of  the  second  model  is  provided  in  Figure  2. 


INSERT  FIGURE  2  ABOUT  HERE 


The  X    parameters  indicates  the  degree  of  correspondence  between 
the  unobservable  constructs  and  the  respective  observable  indicators. 
The  X    parameters  connecting  the  trait  factors  to  the  observations, 
when  squared,  reflect  the  amount  of  variation  due  to  corresponding 
traits.   The  A  parameters  connecting  the  method  to  the  corresponding 
indicators,  when  squared,  reflect  the  amount  of  variation  in  method, 
and  the  error  variance  is  provided  by  9 . . .   If  this  model  fits  the 
data,  it  is  possible  to  conclude  that  convergent  validity  is  achieved 
only  when  method  factors  are  taken  into  account.   For  the  modifications 
of  equations  (1)  and  (2)  to  incorporate  method  factors,  readers  are 
directed  to  Bagozzi  and  Phillips  (1982),  Farh  et  al .  (1984),  Long 
(1983),  and  Phillips  (1981). 

Testing  for  Discriminant  Valdity — Model  3.   If  convergent  validity 
is  achieved  either  through  models  1  or  2,  one  can  proceed  to  assess 
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discriminant  validity.  Discriminant  validity  is  acnieved  wThen  the 
measures  of  each  trait  converge  on  their  corresponding  true  scores 
which  are  unique  from  other  traits.  Stated  differently,  it  is  the 
degree  to  which  a  trait  in  a  theoretical  system  differs  from  other 
traits  in  the  same  theoretical  system.  This  will  be  achieved  when  the 
correlations  between  the  traits  ($  )  are  significantly  lower  than 
unity.   This  requires  a  comparison  of  the  model  in  Figure  2  with  a 

similar  model  in  which  the  three  correlations  are  considered  equal  to 

2 
unity.   A  significantly  lower  x   value  for  the  model  with  the  correla- 

2 
tions  unconstrained  provides  support  for  discriminant  validity.   A  x 

2 
difference  value  (x  j)  value  with  an  associated  p  value  less  than  0.05 

(Joreskog,  1971)  supports  the  discriminant  validity  criterion. 

Testing  for  Association  Between  the  Two  Methods — Model  4.   Although 
we  can  _a  priori  argue  that  the  two  methods  employed  here  are  more  dis- 
similar than  the  two  methods  of  Dess  and  Robinson  (1984) — self-reported 
objective,  and  subjective — the  proposed  testing  system  can  be  used  to 
explicitly  test  the  level  of  association  between  the  two  methods.   Since 
maximally  different  methods  provide  stronger  tests  of  construct  validity 
(Campbell  &  Fiske,  1959),  this  is  an  essential  requirement  for  con- 
struct validation — which  cannot  be  tested  within  the  MTMM  framework.. 

This  test  is  analyzed  in  two  ways — one,  by  testing  the  statistical 

significance  of  the  unconstrained  parameter  <b ,  -  in  Figure  2;  and  the 

■  <-to 

other  by  comparing  the  unconstrained  model  with  a  similar  model  with 
§  ,-  constrained  to  1.   A  significantly  lower  value  of  x   for  the 
unconstrained  model  when  compared  to  the  constrained  model  provides 
further  support  for  the  lack  of  association  between  the  two  methods. 
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DATA 
Primary  Measures 

Primary  measures  of  performance  were  collected  from  senior  execu- 
tives as  a  part  of  a  larger  project  (Venkatraman,  Ramanujam  &  Camillus, 
1984)  between  February  and  April  1984.   Although  the  larger  project  had 
a  response  rate  of  over  33%  (207  cases  out  of  600),  only  86  cases  are 
used  in  this  study.   Since  anonymity  was  assured,  disclosure  of  affi- 
liations was  voluntary.   86  respondents  indicated  their  organizational 
affiliations  which  enabled  us  to  collect  corresponding  secondary  per- 
formance data  on  them. 

The  justification  for  using  the  dimensions — sales  growth,  net 
income  growth,  and  ROI — is  based  on  their  extensive  use  in  strategy 
research  (see  Hofer,  1983  and  Woo  &  Willard,  1983  for  reviews).   More 
specifically,  they  correspond  closely  to  the  indicators  of  Dess  and 
Robinson  (1984),  and  to  the  dimensions  developed  by  Woo  and  Willard 
(1983)  based  on  an  analysis  of  the  data  from  the  PIMS  program. 

Consistent  with  the  relative  nature  of  performance  emphasized  in 
the  strategy  literature,  managers  were  requested  for  their  perceptions, 
not  of  their  absolute  performance,  but  of  their  positions  relative  to 
their  major  competitors.   It  can  be  argued  that  organizations  refer  to 
their  proximate  competitors  rather  than  a  heterogeneous  universe  of 
firms  in  assessing  their  performance.   Relative  performance  was 
obtained  on  a  five-point  scale  ranging  from  -2  (much  worse  than  com- 
petition) to  +2  (much  better  than  competition),  with  the  neutral  point 
of  0  indicating  a  level  equal  to  that  of  competition. 
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Secondary  Measures 

Secondary  measures  of  performance  were  assembled  from  Business 
Week,  magazine's  "Inflation  Scoreboard"  for  the  year  1983,  as  reported 
in  the  March  21,  1984  issue.   Business  Week  compiles  these  data  from 
Standard  &  Poor's  COMPUSTAT   tapes.   Relative  competitive  performance 
was  operationalized  as  firm  performance  relative  to  the  industry, 
where  industry  refers  to  the  principal  SIC  industry  classification  to 
which  the  firm  is  assigned.   Relative  performance  was  measured  as  the 
difference  between  the  values  of  the  indicator  for  the  firm  and  the 
industry.   For  example,  relative  sales  growth  was  the  sales  growth  of 
the  focal  firm  minus  the  growth  rate  for  its  principal  industry. 

RESULTS 
Table  1  presents  the  correlations  among  the  indicators  obtained  as 
discussed  in  the  previous  section.   All  the  analyses  for  model-testing 
were  carried  out  using  the  LISREL  IV  program  (Joreskog  &  Sorbum,  1978). 


INSERT  TABLE  1  ABOUT  HERE 

2 
The  results  of  LISREL  analysis  for  the  first  model  yielded  a  x 

(df:10)  value  of  56.6723;  p=0.00.   This  indicates  that  the  underlying 

hypothesis  that  all  variations  are  due  to  underlying  trait  and  random 

2 
error  only  should  be  rejected.   However,  sole  reliance  on  the  x 

statistic  is  criticized  for  many  reasons  (Fornell  &  Larcker,  1981), 
and  researchers  increasingly  complement  this  statistic  with  two  addi- 
tional statistics.   One  is  the  Bentler  and  Bonnet's  (1980)  incremental 
fit  index  A — which  is  an  indication  of  the  practical  significance  of 
the  model  in  explaining  the  data.   The  A  index  is  represented  as 
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where  F  =  chi-square  value  obtained  from  a  null  model  specifying 
mutual  independence  among  the  indicators,  and  F„  =  chi-square  value 

*      IN. 

for  the  specified  theoretical  model. 

2 
The  other  statistic  is  an  evaluation  of  the  difference  in  x   sta- 
tistic between  two  related  models.   Since  this  was  the  first  model  to 
be  tested,  the  latter  statistic  is  not  appropriate.   But  the  A  index 
was  0.71  lending  further  support  to  the  rejection  of  the  model  since 
as  a  rule  of  thumb  L    should  exceed  0.95  (Bearden,  Sharma,  &  Teel ,  1982) 

This  result  indicates  that  the  second  model  should  be  tested. 

2 
The  analysis  of  the  second  model  yielded  an  overall  statistic  x  (4) 

of  2.97;  p=0.562,  and  the  difference  in  x   between  the  first  and  the 
second  model  was  53.70,  significant  at  p<0.001.   Further,  A  index  of 
0.984  indicated  that  more  than  98%  of  the  measure  variation  is  captured 
by  the  model.   These  results  provide  strong  support  to  the  second  model 
and  the  underlying  hypothesis  that  measures  achieve  convergent  validity 
only  when  the  method  factors  (i.e.,  sources  of  systematic  variation) 
are  explicitly  incorporated  into  the  model. 

Since  convergent  validity  requirements  are  satisfied  in  the  second 
model,  we  can  now  test  for  discriminant  validity  using  model  3.   Dis- 
criminant validity  is  achieved  wnen  the  measures  of  each  dimension 
converge  on  their  corresponding  true  scores  which  are  unique  from  other 
dimensions.   Stated  differently,  it  is  the  degree  to  which  a  dimension 
in  a  theoretical  system  differs  from  other  dimensions  in  the  same 
theoretical  system. 
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2 
The  x   difference  in  the  two  models  (one  with  the  correlations 

between  the  traits  each  constrained  to  be  1.0,  and  the  other  with  these 

2,  2 

correlations  unconstrained),  viz.,  x  (?)  sinus  X  (4)  is  32.56  (35.6324 

minus  2.97);  p<0.001.   This  satisfies  the  criteria  for  discriminant 

validity  (Joreskog,  1971).   However,  the  analysis  of  model  2  indicated 

that  <j>?1  and  $oo  were  large  and  statistically  different  from  zero  (see 

Table  2).   This  could  imply  that  dimensions  1  and  2  and/or  dimensions 

2  and  3  nay  be  subd intensions  of  a  broader  construct. 

In  order  to  rule  out  this  rival  interpretation,  two  separate 

models  were  estimated,  one  with  <?.-,,  constrained  to  1.0,  and  the  other 

2 
with  6-^  constrained  to  1.0.   A  significantly  lower  value  of  x   for 

the  unconstrained  model  indicates  that  tne  dimensions  are  indeed  dif- 

2 
ferent.   The  model  with  £9,  constrained  yielded  a  x   (df :5)  of  26.98; 

? 
and  with  $  ~.~    constrained  yielded  a  x~  (df  :4)  of  2.97,  and  the  difference 

in  x   statistic  in  both  cases  are  significant  at  a  level  better  than 

p=0.01.   This  further  provides  support  for  discriminant  validity  of  the 

three  dimensions  of  3EP. 

The  fourth  model  sought  to  examine  the  association  between  the  two 

method  factors.   The  model  with  $,_  constrained  to  equal  1.0  yielded 

a  statistic  of  x  (•+)  of    17.7676,  while  the  alternate  model  of  uncon- 

?  2 

strained  i>  ,  -    yielded  a  x~(3)  statistic  of  1.088.   The  value  of  y 

difference  with  1  degree  of  freedom  is  16.6788  and  is  significant  at 

p<0.001,  lending  support  to  the  hypothesis  of  dissimilar  methods.   In 

addition,  the  parameter  ©..  when  left  unconstrained  was  0.264  with  a 

corresponding  t-value  of  1.557  which  is  not  significant  at  p<0.01, 

providing  additional  support  to  the  hypothesis  of  dissimilar  methods. 
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Further,  the  LISREL  estimates  can  be  used  to  partition  the  measure- 
ment variance  into  trait,  method,  and  error  components.   Following 
Joreskog  and  Sorbum  (1979)  and  Bagozzi  (1980),  we  decomposed  the 
variance  as  shown  in  Table  2.   Additionally,  measure  reliability  of 
both  individual  indicators  and  composite  index,  calculated  based  on 
the  formulae  derived  in  Werts  et  al .  (1974)  are  included  in  the  table. 


INSERT  TABLE  2  ABOUT  HERE 


'DISCUSSION 
The  support  received  for  convergent  validity  further  corroborates 
the  correspondence  between  secondary  and  primary  sources  of  data  on 
organizational  performance;  and  the  positive  results  in  relative  to 
discriminant  validity  lends  further  credence  to  Woo  and  Willard's 
(1983)  conclusion  regarding  the  multi-dimensional  nature  of  organiza- 
tional performance. 

Support  for  the  Three  Research  Questions 

More  specifically,  the  results  can  be  used  to  address  the  three 
research  questions.   For  the  first  question,  the  results  indicate  that 
both  convergent  and  discriminant  validity  were  achieved  only  when 
method  factors  were  introduced  in  the  model.   It  implies  that  while 
the  MTMM  approach  may  provide  general  indications  of  convergent  and 
discriminant  validity,  additional  analyses  such  as  those  done  here  en- 
ables one  to  examine  specific  reasons  for  the  lack,  of  support  for  various 
validity  criteria  by  partitioning  the  variance  into  its  components. 
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The  partitioning  of  variance  in  Table  2  provides  a  systematic 
basis  for  addressing  the  second  question.   The  average  trait  variance 
explained  by  tne  primary  method  is  45.3%,  while  tne  secondary  method 
explained  43.3%.   The  average  method  variance  for  the  three  dimensions 
is  approximately  equal  with  primary  method  accounting  for  34.3%  and 
secondary  method  for  33.6%;  and  the  random  error  for  the  two  methods 
is  also  similar  (primary  method  -  19%  and  the  second  method  -  20%). 

At  an  aggregate  level,  both  methods  appear  to  be  equally  effective, 
although  tne  ratio  of  trait  variance  to  error  (systematic  +  random) 
variance  is  less  than  1.0,  indicating  poor  measure  reliability.   How- 
ever, viewing  the  dimensions  individually,  some  interesting  results 
can  be  observed.   The  secondary  method  is  more  efficient  (i.e.,  less 
total  error  variance)  for  profitability  (ROI),  while  the  primary  method 
is  more  efficient  for  sales  growth.   The  general  implication  of  the 
results  is  that  managers  are  reasonably  accurate  in  their  perceptions 
of  sales  growth  as  a  performance  measure,  while  they  are  not  as  reliable 
for  profitability  measures. 

Both  methods  appear  to  provide  poor  indications  of  net  income 
growth.   However,  the  variance  partitioning,  in  conjunction  with  the 
analysis  of  measure  reliability  indicate  that  the  use  of  both  methods 
together  is  a  preferred  alternative  for  net  income  growth.   In  other 
cases,  the  composite  appears  to  reduce  the  reliability  due  to  high 
levels  of  measurement  error  for  x2  and  x5  (see  Table  2) .   The  impli- 
cations for  future  research  is  that  while  multiple  methods  should  be 
employed  to  operationalize  the  construct,  only  the  efficient  method 
(or,  methods)  should  be  used  for  testing  substantive  relationships. 
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Further,  we  strongly  advocate  that  tne  superiority  of  one  method  over 
another  should  be  explicitly  tested  as  against  implicitly  assuming  that 
objective  (i.e.,  secondary)  data  are  always  more  "accurate"  than  per- 
ceptual measures. 

For  the  third  question,  the  study  specifically  tested  the  degree 
of  association  between  the  two  methods.   If  methods  are  somewhat  simi- 
lar, such  as  two  different  measuring  instruments  or  self-reported 
objective  and  subjective  data,  it  is  not  difficult  to  establish  con- 
vergent validity  as  demonstrated  in  Dess  and  Robinson  (1984).   Since  in 
this  study  the  two  methods  are  found  to  be  dissimilar,  construct  vali- 
dity assessments  have  a  stronger  impact  than  otherwise.   The  scheme 
also  illustrated  a  systematic  basis  to  identify  poor  quality  indicators 
(e.g.,  x2  and  x5) ,  and  thereby  improve  the  overall  quality  of  measure- 
ment.  Increased  attention  to  measurement  issues  will  certainly  enhance 
the  confidence  which  can  be  placed  on  substantive  research  results. 
Hopefully,  this  paper  will  stimulate  future  strategy  research  studies 
to  address  the  measurement  concerns  raised  in  this  study,  prior  to 
testing  substantive  relationships. 

It  needs  mention  that  although  the  analyses  were  carried  out  using 
the  LISREL  IV  Program  (Joreskog  &  Sorbum,  1978),  it  is  not  the  only 
available  analytical  scheme.   Readers  may  want  to  consider  other  related 
analytical  schemes  such  as  the  partial  least  square  estimation  (Wold, 
1982). 

Limitations  and  Extensions 

Two  limitations  are  noted  with  a  view  to  identifying  future  exten- 
ds.  One  pertains  to  the  size  of  the  sample  employed  for  analysis. 
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Al though  the  sample  size  (an  average  of  80,  after  accounting  for  some 
missing  data)  satisfies  the  minimum  size  for  the  specified  model 
(Bagozzi,  1980;  Lawley  &  Maxwell,  1971),  the  chi-squared  distribution 
is  sensitive  to  sample  size.   However,  by  relying  upon  the  difference 
in  chi-square  (x  j)  statistics  in  the  sense  of  assessing  a  set  of 
nested  models  as  done  here  (which  is  less  sensitive  to  sample  size)  and 
the  use  of  A  index  (Sentler  and  Bonnet,  1980)  which  is  independent  of 
sample  size,  we  attempted  to  reduce  the  problems  associated  with  sample 
size.   Nevertheless,  a  useful  extension  will  be  to  replicate  ttiis  study 
and  test  these  results  using  a  larger  sample  set. 

The  other  issue  pertains  to  the  use  of  single  informant  per  unit 
of  analysis  to  collect  data  on  organization-level  constructs  such  as 
performance.   As  an  extension,  multiple  informants  can  be  used  as 
separate  methods  to  examine  if  systematic  differences  exist  between 
managers  based  on  position,  hierarchy,  and  other  organizational  dif- 
ferences.  It  is  particularly  critical  in  strategy  research  since 
Hambrick's  (1981b)  study  on  strategic  awareness  indicated  a  negative 
association  between  awareness  and  hierarchical  level.   By  employing 
data  collected  from  multiple  levels  and  different  functions  and  decom- 
posing the  method  variance,  useful  guidelines  in  relation  to  research 
design  and  selection  of  respondents  can  be  gleaned. 

SUMMARY 

Using  a  structural  equation  methodology  and  performance  data  on  86 
firms  from  two  different  sources,  this  paper  evaluated  the  construct 
measurement  properties  of  business  economic  performance  measures.   The 
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general  expectation  of  correspondence  between  the  two  methods  was  sup- 
ported, although  both  methods  were  not  equally  efficient  for  measuring 
the  three  dimensions  of  performance.   Implications  for  future  research 
include  an  explicit  evaluation  of  one  method's  superiority  over  another, 
and  combining  methods  when  composite  indices  increases  the  measure 
reliability.   Interesting  differences  from  the  results  obtained  using 
an  MTMM  framework  in  an  earlier  study  (Venkatraman  &  Ramanujam,  1985) 
were  observed.   This  calls  for  researchers  to  examine  the  possible 
adoption  of  the  structural  equation  modeling  methodology  discussed  in 
this  paper  for  addressing  measurement  issues  in  strategic  management 
research. 
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TAiJLE   1 


Zero-order    CorrelaCions    3etween    Indicators 

(n=86) 


XI 

XI 

X2 

X3 

X4 

X5 

X6 

1.000 

X2 

0.442 

1.000 

X3 

0.467 

0.237 

1.000 

X4 

0.332 

0.692 

0.424 

1.000 

X5 

0.364 

0.153 

0.736 

0.333 

1.000 

X6 

0.103 

0.020 

0.357 

0.283 

0.514 

1.000 

— — — — — — 

XI  =  Sales  growth  (primary) 
X2  =  Sales  growth  (secondary) 
X3  =  Net  Income  Growth  (primary) 
X4  =  Net  Income  Growth  (secondary) 
X5  =  Return  on  Investment  (primary) 
X6  =  Return  on  Investment  (secondary) 
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TABLE  2 


Partitioning  of  Variance  and  Measure  Reliability 


DIMENSIONS 

INDICATORS 

VARIANCE  COMPONENTS 

RELIABILITY 

Trait 

Method 

Error 

Individual3 

Composite 

Sales 
Growth 

xl  (P) 
x2  (S) 

0.68 
0.20 

0.12 
0,56 

0.19 
0.20 

0.63 
0.20 

0.60 

Net  Income 
Growth 

x3  (P) 
x4  (S) 

0.33 
0.31 

0.33 
0.31 

0.45 
0.44 

0.33 
0.31 

0.45 

Profit- 
ability 



x5  (P) 
x6  (S) 

0.35 

0.79 



0.46 

0.01 



0.19 
0.20 

0.35 
0.79 

0.72 

For  Individual  Indicators: 


p.  =  X~.Var(A)/{X  .Var(A)  +  Error  variance j 


For  Composite  Measures: 


p      .    =  (EX.)  Var(A)/{(EX.)  Var(A)  +  terror  variance} 
composite      l  l  J 


compos 
where 


P  =  primary  measure; 

S  =  secondary  data  source;  and 

Var(A)  =  variance  of  the  construct,  standardized  at  unity. 


-26- 


FIGURE   1 


a    b 
A  ivlodel    of    Convergent    Validity    With   Only   Trait    Factors    ' 
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X   (df:10)  =  56.6723;  p=0.000. 


$1  ~  53  =  55^ 


and  62  =  64  =  5^,  for 


A  model  without  such  constraints  did  not 
data  (x2(df:10)  =  56.6723;  x2(df:6)  =  54. 
significant).   This  supports  the  model  as 
of  80  was  used  for  estimating  the  model. 


model  identification  purposes, 
provide  a  better  fit  to  the 
3709;  x2d(df:4)  =  2.30— not 
shown;  An  average  sample  size 


In  Figures  1  and  2,  the  notations  of 
are  followed.  Latent  (unobservable)  vari 
are  drawn  as  ellipses;  observable  indicat 
measurement  relations  are  shown  as  arrows 
sented  as  arrows  but  without  origin;  and 
depicted  as  Greek  letters. 


structural  equation  modeling 
ables  or  theoretical  constructs 
ors  are  presented  as  squares; 
;  error  factors  are  also  repre- 
parameters  to  be  estimated  are 
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FiGURE  2 


A  Model  of  Convergent  Validity  With  Trait  and  Method  Factors 


(df:4)    =   2.97;    p=0.562. 


Parameters  ML   Estimates    (t-value    in   parenthesis) 


*1 

*2 

*3 
X4 

*5 
*6 

*7 

*8 

X9 


X 


10 

11 

12 


0.825 
0.444 
0.577 
0.559 
0.593 
0.892 
0.339 
0.675 
0.681 
0.749 
0.661 
-0.138 


(8.620)* 
(4.152)* 
(4.887)* 
(5.343)* 
(5.328)* 
(9.740)* 
(2.869)* 
(7.148)* 
(7.817)* 
(8.410)* 
(6.611)* 
(-1.175)* 


4>2i  =  0.506  (3.891)* 
<J>31  =  0.198  (1.385) 
4>32  =  °'730  (6-793)* 

51  =  63  =  55  =  0.185  (4.072)* 
62  =  64  =  66  =  0.196  (4.523)* 


(*)  -  parameter  significant  at  p<0.05;  t-values  are  calculated  as 
parameter  estimates  divided  by  the  standard  error  of  estimates. 


